The development of novel tools and methods has recently shaped remarkable advances in the field of optical microscopy of biological samples, with particular emphasis in fluorescence imaging (M. Oberholzer, M. Ostreicher, H. Christen&M. Brühlmann, 1996 “Methods in quantitative image analysis”, Histochem. Cell Biol., 105, 333-55). Although fluorescence enables the localization of specific targets with exquisite sensitivity, it presents limitations (M. Tscherepanow, F. Zöliner, M. Hillebrand, &F. Kummert, 2008 “Automatic Segmentation of Unstained Living Cells in Bright-Field Microscope Images” in Advances in Mass Data Analysis of Images and Signals in Medicine, Biotechnology, Chemistry and Food Industry, Springer Berlin Heidelberg, 158-172), such as photobleaching, phototoxicity, and spectral bleed-through due to an increased number of wavelength channels observed. To help minimize these potential constraints, non-fluorescence microscopy modalities are used whenever appropriate, e.g. during evaluation of cellular morphology. Phase contrast and differential interference contrast (DIC) are two widely employed microscopy procedures that effectively render visible unlabeled cells, albeit with associated optical artifacts (the presence of bright and dark halos surrounding cells) that difficult quantitative measurements of cellular shape and size. To overcome fluorescence microscopy limitations, the visualization of unstained samples through bright field microscopy (BFM) has been proposed as an alternative or complementary method to detect, count and/or quantify cell morphology. The use of BFM to test the effects of diverse treatments within cell populations using high-throughput methods is an alternative to the standard cell cytometry approach based on fluorescence microscopy. However, for optically thin and transparent samples such as, but not limited to, unstained cells BFM images lack necessary contrast. This characteristic precludes the use of conventional BFM for cell morphology studies and for high-throughput approaches, as automated segmentation usually requires high-contrast images. To overcome this limitation two approaches have been proposed: the development of segmentation algorithms that work with low-contrast BFM images (M. Tscherepanow, et al., 2008 supra; R. Ali, et al., 2012, “Automatic segmentation of adherent biological cell boundaries and nuclei from brightfield microscopy images”, Mach. Vis. Appl. 23, 607-621; S. Tse, L. Bradbury, J. W. L. Wan, H. Djambazian, R. Sladek, T. Hudson, 2009, “A combined watershed and level set method for segmentation of brightfield cell images”, Proc. SPIE, 7259, Medical Imaging 2009: Image Processing, 72593G) and the development of methods that increase the image contrast so standard segmentation algorithms become feasible. In the present disclosure we focus on strategies that increase image contrast for further processing, through standard segmentation for cell morphology studies and automated high-throughput approaches.
The BFM images carry amplitude and phase information and for optically thin and transparent objects (phase objects) the amplitude component is negligible; this is why BFM images of unstained cells present poor contrast. Furthermore, for phase objects observed under BFM, image contrast is minimal at the focal plane and it increases as the amount of defocus below and above the focal plane increases. The observation of a sample in an out of the focus plane in BFM is known as defocusing microscopy (DM). Agero et al. have shown that a DM image is related to the surface curvature of the object and have used this method to measure the curvature fluctuations at the surface of unstained macrophage cells (U. Agero, L. G. Mesquita, B. R. A. Neves, R. T. Gazzinelli, O. N. Mesquita, 2004, “Defocusing microscopy”, Microsc. Res. Tech., 65, 159-165; J. C. Neto, U. Agero, R. T. Gazzinelli, O. N. Mesquita, 2006, “Measuring Optical and Mechanical Properties of a Living Cell with Defocusing Microscopy”, Biophys. J., 91, 1108-1115). Even when DM increases the contrast of an image allowing a better visualization of the phase object, it also introduces a significant blurring on the image of the object, impeding the use of DM for a quantitative analysis of shape. However, the DM approach has proved effective for automatic cell counting. Drey et al. (L. L. Drey, M. C. Graber, J. Bieschke, 2013, “Counting unstained, confluent cells by modified bright-field microscopy”, Biotechniques, 55, 28-33) have developed a High Contrast (HC) bright field method for cell counting, using DM in conjunction with two optical accessories that increase contrast: a monochromatic filter and a pinhole aperture placed between the condenser and the sample. This methodology is currently used by BioTek Instruments, Inc. to perform microplate-based automated label-free cell counting (Paul Held, Joe Clayton, and Peter Banks, 2016 “High Contrast Bright field Enabling Microplate-based Automated Label-free Cell Counting”, BioTek:Tech Note).
For cell populations with an homogeneous ellipsoidal shape, such as hematopoietic stem cells, Buggenthin et al. (F. Buggenthin, et al., 2013 “An automatic method for robust and fast cell detection in bright field images from high-throughput microscopy”, BMC Bioinformatics, 14, 297) adapted a defocusing microscopy approach with additional digital imaging processing to get a high-throughput, robust, and automated cell counting method from BFM images of unstained cells.
Another approach to address the low contrast of BF images of unstained biological samples is the phase imaging method based on solving the transport of intensity equation (TIE). Methods based on the TIE use a standard BFM and provide a quantitative phase image of the in-focus field of view, producing a high-contrast image suitable for standard segmentation, and have the potential to add 3D shape characterization to the bright field images. Gorthi et al. (S. S. Gorthi, E. Schonbrun, 2012 “Phase imaging flow cytometry using a focus-stack collecting microscope”, Opt. Lett. 37, 707) have implemented this concept into a high-throughput method named Phase imaging flow cytometer (PIFC). PIFC uses a fluid flow to translate the object through different focal planes where a sequence of images are acquired to calculate the derivative of the intensity along the optical axis, and then relate this derivative with the phase image using a second-order partial differential equation deduced from the TIE. Gorthi et al. visualized red blood cells and leukemia cells and used the deduced phase images to know the 3D shape variations of both cell populations. Important drawbacks for this method are: the implementation of the TIE formalism in practice has been difficult and results in phase errors, the selected defocused planes must be optimized (P. K. Poola, V. P. Pandiyan, R. John, 2015 “Quantitative imaging of yeast cells using transport of intensity equation”, IEEE in 2015 Workshop on Recent Advances in Photonics) and the resulting phase image is often affected by low-frequency noise which can obscure images of cells (D. Paganin, A. Barty, P. J. McMahon, K. A. Nugent, 2004 “Quantitative phase-amplitude microscopy. III. The effects of noise”, J. Microsc., 214, 51-61).
A different digital processing approach to improve image quality and to increase image contrast in bright field is disclosed in U.S. Pat. No. 8,744,164 B2 and in J. Selinummi, et al. 2009, “Bright Field Microscopy as an Alternative to Whole Cell Fluorescence in Automated Analysis of Macrophage Images”, PLoS One, 4, e7497, where a bright field image stack is first acquired by defocusing the sample through evenly-spaced increments and then the intensity variation between images (image stack variance) is computed. They proved that the resulting two-dimensional projection image has increased contrast and it can be segmented through standard algorithms developed for fluorescent microscopy (available in the freely-distributed software CellProfiler). Cell morphology obtained through the intensity projection method matches the morphology of the same cells found by fluorescent microscopy, validating the effectiveness of the intensity projection method. Some drawback of this method are: the need to rapidly acquire at least three images at different focal planes (before cells present any cell movement or morphological change), the simultaneous acquisition of bright field and fluorescence images for automated segmentation of bright field images (for cells with heterogeneous shapes such as macrophages), and finally for cells such as yeast U.S. Pat. No. 8,744,164 B2 mentions that important halo effects in bright field images are emphasized erroneously with this intensity projection method.
Adiga et al. (U. Adiga, et al., 2012 “Automated Analysis and Classification of Infected Macrophages Using Bright-Field Amplitude Contrast Data”, J. Biomol. Screen., 17, 401-408), US2017/052106A1 and WO2015/168026A2 have proposed high-throughput methods for classification of unstained cell populations using machine learning-based methods. Adiga et al. increased BFM images contrast by extending the camera exposure time, and using a digital contrast enhancement step followed by background subtraction. An improved segmentation algorithm is then performed and finally they implemented a machine learning algorithm for the automated classification of different cell populations, based on thousands of parameters obtained from the bright field image. The method of US2017/052106A1 does not have a contrast enhancement step; instead, they use BFM images and dark field microscopy images together with a standard segmentation step performed on the freely-distributed software CellProfiler to feed their machine learning algorithm to classify cells according to stage in the cell cycle or cell type.
The use of deconvolution as a method to increase image contrast has been proposed. Deconvolution is an image restoration method, well established for fluorescent microscopy, which reduces the effect of out-of-focus light to yield a sharper image. Deconvolution is performed through iterative algorithms, where the point spread function (PSF) is provided either as a measured PSF or as a theoretical PSF. Alternatively, in blind deconvolution the PSF does not need to be known and it is obtained as an output of the iterative algorithm, together with the final restored image.
For BFM the implementation of deconvolution algorithms has been scarce due to the experimental difficulties of measuring the PSF, limited by low signal and poor contrast produced by sub-diffraction sized beads used in these measurements. In addition, the PSF in BFM possesses an amplitude (aPSF) and a phase (pPSF) component, thus standard deconvolution algorithms (such as those used in fluorescence microscopy) cannot be applied unless one of the two BFM PSF components is neglected. Typically, thin samples are stained for observation in bright field, and therefore only the amplitude component of the PSF (aPSF) is considered.
P. J. Tadrous, 2010 (“A method of PSF generation for 3D brightfield deconvolution”, J. Microsc., 237, 192-199), pointed out the difficulties to propose a theoretical PSF for BFM, and he presents a method to estimate the bright field PSF based on deconvolution of a measured z-stack from a thin sample and starting with an idealized PSF. However, this methodology works only with high-contrast thin samples, which implies a staining step. Holmes et. al., point out the experimental difficulties of measuring the PSF in BFM, so they proposed a blind deconvolution method where the PSF does not need to be known to obtain a restored image (T. J. Holmes, N. J. O'Connor, 2000 “Blind deconvolution of 3D transmitted light brightfield micrographs”, J. Microsc., 200, 114-27). However, their blind deconvolution algorithm is also restricted to absorbing (stained) samples, where the phase component of the image and the phase component of the PSF are both ignored to simplify the deconvolution algorithm.
Image analysis software programs such as CellProfiler or Huygens Software by Scientific Volume Imaging allow to perform deconvolution on bright field images, as long as the images present high contrast, so the deconvolution process offered by these software programs are not suitable for phase objects such as unstained cells.
Recently, Hernandez-Candia and Gutiérrez-Medina (C. N. Hernandez Candia, B. Gutiérrez-Medina, 2014 “Direct Imaging of Phase Objects Enables Conventional Deconvolution in Bright Field Light Microscopy”, PLoS One, 9, e89106) have used computer-enhanced bright field microscopy (CEBFM) (B. Gutiérrez-Medina, S. M. Block, 2010 “Visualizing individual microtubules by bright field microscopy”, Am. J. Phys., 78, 1152-1159) with and improved image background acquisition step to get, for the first time, the measured pPSF of a BFM with a high signal-to-noise ratio. They proposed a phenomenological model for the pPSF that was in excellent agreement with measurements, and as a proof of principle they applied standard deconvolution to CEBFM images of unstained cells. Restored images of bacteria cells showed an increased contrast with a well-defined cell boundary, fully removing the halo of bright-dark rings characteristic of bright field images. Unlike previous BFM deconvolution methods which are restricted to high contrast images, the work of Hernandez-Candia et al. is suitable for phase objects, such as unstained cells, which produced a low contrast image.
The disclosure presented herein also uses deconvolution as a means to restore images of unstained samples, increasing their contrast. However, unlike Hernandez-Candia et. al. in the herein invention deconvolution is performed from a theoretical pPSF, and the image restoration procedure is validated by showing that quantitative analysis of sample size and shape in the deconvolved bright field image and in a reference fluorescence image are in agreement. In addition, it is proved that quantitative analysis of sample morphology can be performed from two dimensional (2D) deconvolution of a single input frame, and that BF digital restoration of unstained samples can be used in high-throughput image cytometry approaches.